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Highlights 

Highlights  of  GAO-15-421,  a  report  to  the 
Committee  on  Armed  Services,  U.S.  Senate 


DOD  RAPID  INNOVATION  PROGRAM 

Some  Technologies  Have  Transitioned  to  Military 
Users,  but  Steps  Can  Be  Taken  to  Improve  Program 
Metrics  and  Outcomes 


Why  GAO  Did  This  Study 

DOD  relies  on  technology  innovation  to 
maintain  superior  weapon  systems. 
However,  a  long-standing  challenge 
has  been  ensuring  that  high  value 
technologies  are  mature  and  available 
for  military  users.  The  Ike  Skelton 
National  Defense  Authorization  Act  for 
Fiscal  Year  201 1  required  DOD  to 
establish  RIP  to  facilitate  the  transition 
of  innovative  technologies  into 
acquisition  programs,  and  over  $1 .3 
billion  has  been  appropriated  to  the 
program  since  its  inception. 

Senate  Report  No.  113-44  included  a 
provision  that  GAO  review  the 
execution  of  RIP.  This  report  examines 
the  extent  to  which  (1)  DOD  has 
established  a  competitive,  merit-based 
process  to  award  RIP  contracts;  (2) 
DOD  has  established  practices  to 
manage  the  execution  of  RIP  projects; 
and  (3)  RIP  is  meeting  its  objective  of 
rapidly  inserting  innovative 
technologies  in  defense  acquisition 
programs.  GAO  reviewed  RIP 
documentation,  interviewed  DOD  and 
military  department  officials,  and 
reviewed  a  nongeneralizable  sample  of 
40  projects  awarded  with  fiscal  year 
201 1  and  2012  funding  to  assess  DOD 
management  practices  as  well  as  52 
fiscal  year  2011  funded  projects 
scheduled  for  completion  through  July 
2014  to  assess  transition  outcomes. 


What  GAO  Found 

The  Department  of  Defense  (DOD)  has  established  a  competitive,  merit-based 
process  to  solicit  proposals  from  interested  contractors,  review  and  select 
projects  based  on  military  needs  and  standardized  evaluation  criteria,  and  award 
contracts  to  execute  Rapid  Innovation  Program  (RIP)  projects.  To  date,  the 
process  has  been  lengthy,  taking  about  18  months  to  implement,  but  interest 
from  contractors  has  been  high.  Between  fiscal  years  2011  and  2015,  the  military 
services  and  defense  agency  components  received  more  than  11,000  white 
papers  on  proposed  technologies  from  contractors  and  will  have  awarded 
contracts  for  about  435  projects — primarily  to  small  businesses — when  the  fiscal 
year  2014  solicitation  is  completed. 

The  military  services  and  defense  components  have  practices  and  tools  in  place 
to  manage  and  monitor  the  execution  of  projects,  which  are  similar  to  those  they 
use  for  other  science  and  technology  projects.  For  example,  project  officials 
review  contractor  reports,  conduct  system  reviews,  and  engage  in  regular 
communications  with  contractors  to  determine  the  progress  of  projects.  Also, 
DOD's  2014  annual  review  found  that  85  percent  of  the  fiscal  year  2011  funded 
projects  and  78  percent  of  the  fiscal  year  2012  funded  projects  were  likely  to 
meet  80  percent  or  more  of  their  technical  performance  goals. 

Some  completed  projects  have  successfully  transitioned  technology  to 
acquisition  programs  and  other  military  users.  DOD  officials  estimate  that  50 
percent  of  all  fiscal  year  2011  funded  projects  (88  of  175)  have  out-year  funding 
commitments  from  military  users,  indicating  the  likelihood  they  will  transition 
technologies.  GAO  assessed  44  projects  completed  in  July  2014,  and  found  that 
50  percent  successfully  transitioned  to  acquisition  programs  or  other  users. 
However,  it  is  too  soon  to  accurately  assess  the  overall  success  of  RIP  due  to 
the  limited  number  of  completed  projects  as  well  as  the  lack  of  an  overall 
program  transition  goal  and  effective  metrics  to  track  the  degree  to  which 
projects  have  actually  transitioned.  GAO  found  that  several  factors  can  contribute 
to  transition  success  of  RIP  projects,  such  as  having  military  user  commitment 
and  mature  technology  when  projects  are  started.  However,  DOD  has  not  made 
an  effort  to  understand  how  these  factors  may  be  contributing  to  differences  in 
transition  success  from  defense  components  with  a  higher  rate  of  transition. 


What  GAO  Recommends 

GAO  recommends  that  DOD  establish 
a  program  transition  goal,  and  identify 
and  apply  factors  that  lead  to  transition 
success.  DOD  disagreed  on  the  need 
for  a  goal,  stating  it  would  impede  RIP 
flexibility,  but  agreed  to  identify 
transition  success  factors.  GAO 
believes  having  a  goal  would  improve 
DOD’s  ability  to  transition  technologies. 

View  GAO-15-421.  For  more  information, 
contact  Michele  Mackin  at  (202)  512-4841  or 
mackinm@gao.gov. 
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Abbreviations 


ASD/R&E 

Assistant  Secretary  of  Defense  for  Research  and 
Engineering 

BAA 

Broad  Agency  Announcement 

DOD 

Department  of  Defense 

FAR 

Federal  Acquisition  Regulation 

GMTI 

Ground  Moving  Target  Indicator 

OSD 

Office  of  the  Secretary  of  Defense 

RIP 

Rapid  Innovation  Program 

S&T 

Science  and  technology 

SBIR 

Small  Business  Innovation  Research 

SOCOM 

United  States  Special  Operations  Command 

TRL 

Technology  Readiness  Level 
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GAO 

441  G  St.  N.W. 
Washington,  DC  20548 


U.S.  GOVERNMENT  ACCOUNTABILITY  OFFICE 


May  7,  2015 

The  Honorable  John  McCain 
Chairman 

The  Honorable  Jack  Reed 
Ranking  Member 
Committee  on  Armed  Services 
United  States  Senate 

The  Department  of  Defense  (DOD)  relies  on  technological  innovation  to 
maintain  the  superiority  of  its  weapon  systems  and  armed  forces  to 
protect  U.S.  interests  at  home  and  abroad.  However,  a  long-standing 
challenge  for  DOD  has  been  ensuring  that  high  value  technologies  from 
defense  research  organizations  and  businesses  are  mature  and  available 
for  use  by  the  department’s  acquisition  community.  Sometimes 
technologies  are  not  ready  to  transition  to  a  defense  acquisition  program 
when  needed  because  they  may  still  be  too  risky  or  too  costly  to  adopt.  At 
other  times,  promising  technologies  are  not  taken  advantage  of  due  to 
insufficient  processes  and  mechanisms  to  expedite  their  transition  to 
military  users.  DOD  and  Congress  recognize  these  as  well  as  other 
problems  in  transitioning  technology  and  have  initiated  a  number  of 
programs  over  the  past  decade  aimed  at  removing  transition  barriers  and 
accelerating  the  flow  of  innovative  technologies  to  users.  The  Ike  Skelton 
National  Defense  Authorization  Act  for  Fiscal  Year  201 1  required  DOD  to 
establish  the  Rapid  Innovation  Program  (RIP)  to  facilitate  the  insertion  of 
innovative  technologies,  into  defense  acquisition  programs.1  Between 
fiscal  years  2011  and  2015,  Congress  appropriated  about  $1.3  billion  to 
fund  technology  projects  through  the  program. 

The  report  accompanying  Senate  Bill  1197,  National  Defense 
Authorization  Act  for  Fiscal  Year  2014,  included  a  provision  that  GAO 
report  on  the  execution  of  the  Rapid  Innovation  Program.2  The  objectives 
of  this  review  were  to  assess  the  extent  to  which  (1 )  DOD  has  established 
a  competitive  and  merit-based  process  to  solicit  and  award  RIP  contracts, 
(2)  DOD  has  established  practices  to  manage  and  oversee  the  execution 


1Pub.  L.  No.  111-383,  §  1073. 

2S.  Rep.  No.  1 1 3-44  at  54-56  (2013). 
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of  RIP  projects,  and  (3)  RIP  is  meeting  its  objective  of  rapidly  inserting 
innovative  technologies  into  defense  acquisition  programs. 

To  conduct  this  work,  we  evaluated  documentation  from  the  Under 
Secretary  of  Defense  for  Acquisition,  Technology,  and  Logistics;  the  Air 
Force;  the  Army;  Navy;  Special  Operations  Command;  and  other  defense 
agencies  involved  in  RIP;  and  interviewed  key  program  and  project 
officials.  To  assess  how  DOD  solicited  and  awarded  RIP  projects,  we 
reviewed  DOD  and  military  department  polices  and  guidance  on  the 
program  as  well  as  solicitation  and  source  selection  documents.  Also,  we 
interviewed  officials  responsible  for  developing  the  guidance  and  those 
involved  in  the  source  selection  process.  To  assess  how  DOD  manages 
and  oversees  RIP  projects,  we  reviewed  a  non-generalizable  random 
sample  of  40  RIP  projects  across  DOD  that  were  awarded  with  fiscal  year 
201 1  or  2012  funding.  We  selected  this  time  frame  to  capture  projects 
that  would  be  at  or  near  completion.  In  addition,  we  reviewed  project 
contracts,  contractor  reports,  agency  project  assessments,  and 
interviewed  officials  responsible  for  project  oversight  and  management. 

To  determine  whether  the  RIP  is  meeting  its  goal  of  rapidly  inserting 
innovative  technologies  into  acquisition  programs,  we  reviewed  available 
program  monitoring  information  and  assessed  the  transition  status  of  all 
projects  that  were  scheduled  for  completion  by  the  end  of  July  2014.  This 
totaled  52  projects  awarded  with  fiscal  year  201 1  funding.3  For  these 
projects,  we  interviewed  DOD  RIP  project  officials  and  collected  and 
analyzed  documents  outlining  processes  and  procedures  used  to  manage 
the  projects  and  promote  transition  opportunities.  We  discussed  factors 
that  may  have  helped  or  hindered  project  execution  and/or  successful 
transition  to  defense  acquisition  programs  or  other  military  users.  We  also 
used  data  from  DOD’s  annual  in-process  review  on  the  status  and 
progress  of  RIP  projects,  and  determined  through  discussions  with  DOD 
subject  matter  experts  that  the  data  were  sufficiently  reliable  for  the 
purposes  of  this  report.  See  appendix  I  for  additional  details  on  the  scope 
and  methodology  used  for  this  work. 

We  conducted  this  performance  audit  from  April  2014  to  May  2015  in 
accordance  with  generally  accepted  government  auditing  standards. 
Those  standards  require  that  we  plan  and  perform  the  audit  to  obtain 


3Although  Congress  first  appropriated  funds  for  RIP  in  fiscal  year  201 1 ,  projects  were  not 
awarded  until  the  end  of  fiscal  year  2012  and  then  not  expected  to  be  completed  until  two 
years  after  contract  award,  which  was  the  end  of  fiscal  year  2014. 


Page  2 


GAO-15-421  Rapid  Innovation  Program 


sufficient,  appropriate  evidence  to  provide  a  reasonable  basis  for  our 
findings  and  conclusions  based  on  our  audit  objectives.  We  believe  that 
the  evidence  obtained  provides  a  reasonable  basis  for  our  findings  and 
conclusions  based  on  our  audit  objectives. 


Background 


To  identify,  pursue,  and  develop  new  technologies  to  improve  and 
enhance  military  capabilities,  DOD  relies  on  its  science  and  technology 
(S&T)  community  which  is  comprised  of  DOD  research  laboratories,  test 
facilities,  industry,  and  academia.  The  S&T  community  receives  about 
$12  billion  in  funding  each  year  to  support  activities  ranging  from  basic 
research  through  advanced  technology  development  that  are  conducted 
by  the  government  or  externally  by  universities  and  commercial  industry. 
Once  the  S&T  community  has  completed  its  technology  development, 
additional  product  development  activities,  such  as  technology 
demonstration  and  testing,  are  often  needed  before  incorporating  the 
technologies  into  military  weapon  systems.  Under  the  management  of  the 
acquisition  community,  product  development  further  advances  technology 
received  from  S&T  developers  and  integrates  it  into  systems  that  are 
ultimately  delivered  to  support  the  warfighter  (see  figure  1 ). 


Figure  1:  DOD  Technology  Management 


Technology  Investment 


Technology  Development 


Product  Development 


Technology  shelved 
but  available  for  use 


Technology 
integrated  into 
acquisition 
program  or  fielded 


Source:  GAO  analysis  of  DOD  S&T  management  process.  |  GAO-15-421 
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However,  as  we  have  reported  in  the  past,  for  a  variety  of  reasons  DOD 
historically  has  experienced  problems  in  transitioning  technologies  out  of 
the  S&T  environment  and  into  military  systems.4  For  example, 
technologies  may  not  leave  the  laboratory  because  their  potential  has  not 
been  adequately  demonstrated  or  recognized,  acquisition  programs  may 
be  unwilling  to  fund  final  stages  of  development,  or  private  industry 
chooses  to  develop  the  technologies  itself. 

DOD  has  a  variety  of  technology  transition  programs  managed  by  the 
Office  of  the  Secretary  of  Defense  (OSD)  and  the  military  departments 
that  provide  mechanisms  and  funding  to  facilitate  technology  transitions. 
The  programs  vary  in  size,  approach,  and  funding,  but  most  of  them  are 
intended  to  target  fairly  mature  technologies  that  are  suitable  for  the  final 
stages  of  development  and  demonstration.  Some,  such  as  the  Small 
Business  Innovation  Research  (SBIR)  program,  have  a  stated  purpose  of 
using  small  businesses  to  meet  federal  research  and  development 
needs.5  However,  while  the  program  provides  funding  for  technology 
feasibility  and  development  projects,  small  businesses  are  expected  to 
obtain  funding  from  the  private  sector  or  government  sources  outside  the 
SBIR  program  to  commercialize  or  transition  technologies  for  sale  to  the 
military  or  elsewhere. 

Congress  established  RIP  as  another  mechanism,  to  include 
consideration  of  innovation  research  projects  from  small  businesses,  as 
they  are  often  viewed  as  a  key  source  for  developing  innovative 
technologies  in  areas  of  military  need.  Specifically,  Congress  directed 


4GAO,  Defense  Technology  Development:  Technology  Transition  Programs  Support 
Military  Users,  but  Opportunities  Exist  to  Improve  Measurement  of  Outcomes, 

GAO-13-286  (Washington,  D.C.:  Mar.  7,  2013);  and  GAO  Best  Practices:  Stronger 
Practices  Needed  to  Improve  DOD  Technology  Transition  Processes,  GAO-06-883 
(Washington,  D.C.:  Sept.  14,  2006). 

5ln  1982,  Congress  established  the  Small  Business  Innovation  Research  (SBIR)  program 
to  stimulate  technological  innovation  and  to  utilize  small  businesses  for  federal  research 
and  development  needs.  Pub.  L.  No.  97-219.  To  be  eligible  to  participate,  a  company 
must  be  a  U.S. -owned  business  with  500  or  fewer  employees.  The  program  is  structured 
into  three  phases,  with  projects  progressing  from  initial  technology  feasibility,  to 
technology  development  and  prototyping,  and  to  commercialization.  The  program  provides 
up  to  $1.15  million  for  phases  one  and  two.  For  additional  information  on  DOD’s  SBIR 
program  see  GAO,  Small  Business  Innovation  Research:  DOD's  Program  Supports 
Weapon  Systems,  but  Lacks  Comprehensive  Data  on  Technology  Transition  Outcomes , 
GAO-14-96  (Washington,  D.C.:  Dec.  20,  2013). 
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DOD  to  establish  RIP  to  accelerate  the  transition  of  technologies 
developed  by 

•  small  businesses  participating  in  SBIR  phase  II  projects; 

•  the  defense  laboratories;  and 

•  other  small  or  large  businesses. 

The  projects  funded  by  RIP  are  to  primarily  support  major  defense 
acquisition  programs,  but  also  other  defense  acquisition  programs  that 
meet  a  critical  national  security  need.  In  addition,  the  projects  selected 
are  to  accelerate  the  fielding  of  technologies  with  the  purpose  of  reducing 
acquisition  or  lifecycle  costs;  addressing  technical  risks;  and  improving 
test  and  evaluation  outcomes.  Furthermore,  DOD  was  directed  to  develop 
a  competitive,  merit-based  program  that  at  a  minimum  provides  for 

•  the  use  of  a  broad  agency  announcement  or  the  use  of  any  other 
competitive  or  merit-based  processes  for  the  solicitation  of  proposals; 

•  merit-based  selection  of  the  most  promising  cost-effective  proposals 
for  funding  through  contracts,  cooperative  agreements,  and  other 
transactions; 

•  a  limit  on  funding  for  each  RIP  project  not  exceeding  $3  million;  and 

•  no  project  is  funded  under  the  program  for  more  than  2  years.6 


Within  DOD,  the  Under  Secretary  of  Defense  for  Acquisition,  Technology, 
and  Logistics  issues  overall  program  guidelines,  and  representatives 
appointed  by  the  military  service  acquisition  executives,  Assistant 
Secretary  of  Defense  for  Research  and  Engineering  (ASD/R&E),  and  the 
Director,  Office  of  Small  Business  Programs  have  the  responsibility  for 
establishing  the  RIP  processes  that  support  DOD’s  goal  and  meet  the 
guidelines.  The  ASD/R&E  is  also  responsible  for  coordinating  RIP 
activities  among  the  military  departments  and  other  defense  components. 
In  addition,  ASD/R&E  is  responsible  for  preparing  and  submitting  a  report, 


6However,  the  Secretary  of  Defense  or  the  Secretary’s  designee  may  approve  a  larger 
funding  amount  for  a  project  or  may  approve  funding  for  any  additional  year. 
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at  the  end  of  the  fiscal  year,  to  the  congressional  defense  committees  that 
includes  a  list  and  description  of  each  project  funded,  the  amount  of 
funding  provided,  and  anticipated  timeline  for  transition.  The  program  is 
implemented  and  managed  by  each  of  the  military  departments  and  by 
ASD/R&E,  which  represents  the  other  defense  components  that 
participate  in  the  program.7  To  date,  DOD  has  not  included  RIP  in  its 
annual  budget  requests  because  there  is  no  formal  requirement  within  the 
department  for  the  program,  but  Congress  has  appropriated  funding  each 
year.  Congress  has  authorized  RIP  until  September  30,  2015,  when  it  will 
expire  unless  further  legislative  action  is  taken. 


DOD  Uses  a 
Competitive  and 
Merit-Based  Process 
to  Select  RIP 
Projects,  with  Most 
Awards  Going  to 
Small  Businesses 


DOD  has  established  a  competitive,  merit-based  solicitation  process  to 
select  and  award  RIP  contracts  that  address  military  needs.  Each  year, 
the  Office  of  the  Under  Secretary  of  Defense  for  Acquisition,  Technology, 
and  Logistics  begins  by  issuing  broad  guidance  to  the  military 
departments  and  defense  components  for  implementing  a  multi-phase 
process  to  identify  military  needs,  solicit  proposals  from  interested 
contractors,  review  and  select  projects,  and  award  contracts.  The  process 
is  somewhat  lengthy,  taking  about  18  months  to  implement  and  award 
contracts,  but  interest  from  contractors  has  been  high.  Between  fiscal 
year  201 1  and  2015,  the  military  departments  and  defense  components 
received  more  than  1 1 ,000  white  papers  from  interested  contractors  and 
will  have  awarded  contracts  for  about  435  projects  when  the  fiscal  year 
2014  solicitation  is  completed,  with  the  vast  majority  going  to  small 
business.  This  high  level  of  interest  in  the  program  has  presented  some 
administrative  challenges  to  reducing  the  time  from  identification  of  need 
to  contract  award. 


DOD  Has  Implemented  a  DOD  has  established  a  multi-step  process  to  solicit,  evaluate,  and  select 
Multi-Step  Process  to  RIP  projects  to  fund  (see  figure  2). 

Select  Promising  RIP 
Technology  Projects 


7For  purposes  of  this  report,  the  term  “defense  component”  refers  to  defense  agencies 
(e.g.,  Missile  Defense  Agency,  Defense  Threat  Reduction  Agency)  and  combatant 
commands  (e.g.,  Special  Operations  Command). 
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Figure  2:  Rapid  Innovation  Program  Project  Selection  Process 


Planning  Phase 


White  Paper  Phase 


Proposal  Phase 


Award  Phase 


Source:  GAO  analysis  of  DOD  data.  |  GAO-15-421 


Before  the  yearly  RIP  acquisition  cycle  begins,  the  Office  of  the  Under 
Secretary  of  Defense  for  Acquisition,  Technology,  and  Logistics  issues 
implementation  guidelines  to  the  military  departments  and  defense 
components  which  outline  program  goals  and  specific  guidance  on  the 
program’s  implementation  and  reporting  requirements.  For  example,  the 
guidelines  include  the  funding  available  to  the  military  departments  and 
defense  components;  DOD-wide  research  areas,  if  applicable;  guidance 
on  how  the  solicitation  process  will  be  structured,  such  as  whether  a 
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Identification  of  Technology 
Needs 


single  solicitation  will  be  used  and  the  length  of  time  it  will  be  open;  and 
the  process  for  obtaining  a  waiver  for  projects  expected  to  exceed  the  24 
month  or  $3  million  limits. 

The  RIP  process  begins  with  the  military  departments  and  defense 
components  identifying  the  technology  topics  that  will  be  addressed  in  the 
program.  In  addition  to  the  3  military  departments,  participation  by 
defense  components  has  increased,  from  4  in  fiscal  year  201 1  to  16  in 
fiscal  year  2014.  The  topics  are  intended  to  reflect  technology  capability 
needs  and  requirements  for  weapon  systems  and  acquisition  programs, 
and  are  one  of  the  criteria  used  in  the  selection  of  projects.  The  military 
departments  and  defense  components  have  used  different  approaches  to 
identify  the  technology  research  topics,  but  over  time  have  moved  to  a 
more  common  approach.  For  example,  while  the  Air  Force  and  Navy 
have  largely  relied  on  their  acquisition  communities  to  identify  technology 
needs,  the  Army  relied  primarily  on  its  science  and  technology  community 
to  identify  needs  for  the  first  3  years  of  the  program.  For  the  Air  Force  and 
Navy,  this  entailed  going  directly  to  systems  commands,  Program 
Executive  Offices,  program  managers,  and  others  directly  involved  in  the 
development  and  production  of  weapon  systems  to  solicit  needs.  In 
relying  on  its  science  and  technology  community,  the  Army  focused  on 
S&T  research  areas  that  included  more  than  the  specific  technology 
needs  of  acquisition  programs.  In  fiscal  year  2014,  the  Army  expanded  its 
approach  and  solicited  topics  from  Program  Executive  Offices  as  well  as 
from  the  science  and  technology  community.  The  defense  components 
used  a  variety  of  methods  for  identifying  technology  requirements.  For 
example,  the  Missile  Defense  Agency  develops  research  topics  by 
reviewing  technology  road  maps  and  prior  SBIR  research  topics,  and 
obtaining  input  from  Program  Executive  Officers  and  program  managers. 
The  combatant  command  participants,  such  as  the  United  States  Special 
Operations  Command  (SOCOM),  include  operational  capabilities  linked  to 
internal  requirements  or  urgent  needs. 
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Competitive,  Merit-Based 
Process  Used  to  Solicit  and 
Award  Projects 


The  military  departments  and  defense  components  have  used  broad 
agency  announcements  (BAA)  to  solicit  technical  solutions  that  address 
technology  topics  and  meet  other  submission  requirements.8  The  Federal 
Acquisition  Regulation  (FAR)  provides  that  the  selection  of  basic  and 
applied  research  is  a  competitive  procedure  if  an  award  results  from  (1)  a 
broad  agency  announcement  that  is  general  in  nature,  identifying  areas  of 
research  interest,  including  criteria  for  selecting  proposals,  and  soliciting 
the  participation  of  all  offerors  capable  of  satisfying  the  government’s 
needs;  and  (2)  a  peer  or  scientific  review.  The  FAR  also  requires  that 
BAAs  specify  the  period  of  time  for  parties  to  respond,  and  contain 
instructions  for  preparing  and  submitting  proposals.9  Also,  we  have  found 
in  a  prior  review  that  the  use  of  competitive  contracting  procedures 
encourages  firms  to  offer  their  best  proposals  when  competing  for  work.10 
The  procedures  used  by  the  military  departments  and  defense 
components  for  selecting  RIP  projects  generally  follow  FAR  competition 
guidelines  because  the  BAAs 

•  identified  general  areas  of  research  interest  and  criteria  for  selecting 
proposals; 

•  solicited  the  participation  of  all  offerors  capable  of  satisfying  the 
government’s  needs;  and 

•  that  subject  matter  experts  will  review  project  proposals. 

For  fiscal  years  201 1  to  201 3,  the  military  departments  and  the  ASD/R&E 
published  four  separate  BAAs,  although  they  used  similar  selection  and 
evaluation  criteria.  In  fiscal  year  2014,  a  single  BAA  was  used  to  solicit 
proposals  because  officials  wanted  to  make  it  easier  for  businesses  to 


8A  BAA  is  a  general  announcement  of  an  agency’s  research  interest,  including  criteria  for 
selecting  proposals,  and  soliciting  the  participation  of  all  offerors  capable  of  satisfying  the 
government’s  needs.  Federal  Acquisition  Regulation  (FAR)  §§  2.101;  6.102(d)(2).  The 
BAA  must  be  publicized  through  the  government-wide  point  of  entry,  the  Federal  Business 
Opportunities  website  (www.FedBizOpps.gov),  and  under  certain  circumstances  it  may 
also  be  published  in  noted  scientific,  technical,  or  engineering  periodicals.  FAR 
§35.016(c). 

9FAR§  6.102(d)(2). 

10GAO,  Pension  Benefit  Guaranty  Corporation:  More  Strategic  Approach  to  Contracting 
Still  Needed,  GAO-1 1-588  (Washington,  D.C.:  Jun.  29,  2011). 
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participate  in  the  program  and  reduce  some  of  the  internal  overhead 
involved  in  having  to  maintain  four  separate  BAAs  within  DOD. 

DOD  has  established  a  merit-based  review  process  to  select  projects 
through  a  two-step  approach — submission  and  review  of  brief  white 
papers  submitted  by  contractors,  then  comprehensive  technical  proposals 
from  a  subset  of  these  contractors  whose  white  papers  are  deemed  most 
promising  based  on  four  criteria: 

•  Contribution  to  Requirement — the  degree  to  which  the  technical 
approach  is  relevant  to  the  proposed  requirement; 

•  Technical  Approach  and  Qualifications — the  degree  to  which  the 
technical  approach  is  innovative,  feasible,  achievable,  complete  and 
supported  by  a  technical  team  that  has  the  expertise  and  experience 
to  accomplish  the  proposed  tasks; 

•  Schedule — the  degree  to  which  the  proposed  schedule  is  achievable 
within  24  months  from  contract  award;  and 

•  Cost — the  degree  to  which  the  proposed  cost  or  price  is  realistic  for 
the  proposed  technical  approach  and  does  not  exceed  $3  million. 

According  to  RIP  officials,  this  approach  is  viewed  positively  by  small 
businesses  because  they  do  not  have  to  invest  in  developing  full 
proposals  at  the  outset. 

Evaluation  teams  of  subject  matter  experts,  nominated  by  the  military 
departments  and  defense  activities,  rate  the  white  papers  “go”  or  “no  go” 
based  on  the  criteria  above.  Greater  consideration  is  given  to  the  criteria 
pertaining  to  contribution  to  requirement  and  technical  approach;  if  papers 
do  not  meet  either  of  these  two  criteria  they  are  not  considered  for  further 
review.  The  contractors  submitting  white  papers  that  are  approved  by  the 
source  selection  authority  for  further  consideration  are  then  invited  to 
submit  a  full  proposal.11  According  to  a  DOD  official,  more  white  papers 


11  The  source  selection  authority  is  the  official  designated  to  make  the  source  selection 
decision.  The  military  departments  are  the  source  selection  authority  for  their  projects. 
Assistant  Secretary  of  Defense  for  Research  and  Engineering  and  the  Director  of  Office  of 
Small  Business  Programs  serve  as  the  review  authority  responsible  for  prioritizing  projects 
submitted  by  the  defense  components  that  participate  in  RIP.  The  defense  components 
are  the  source  selection  authorities  for  their  approved  projects. 
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are  approved  than  can  be  funded  because  some  technologies  will  not 
remain  competitive  when  further  details  and  support  are  provided  in  a  full 
proposal. 

Once  businesses  are  notified  that  their  white  papers  have  been  selected, 
they  have  30  days  to  submit  full  proposals  that  amplify  the  information  in 
the  white  paper.  The  military  departments  and  defense  components 
assign  subject  matter  evaluators  to  review  the  proposals  using  the  same 
criteria  as  for  the  white  paper  evaluations.  DOD  uses  a  5-level  descriptive 
rating  scale,  ranging  from  outstanding  to  unacceptable,  when  evaluating 
the  proposals  against  the  criteria.  As  with  the  white  paper  evaluations, 

RIP  gives  greater  importance  to  obtaining  superior  technical  capabilities 
that  will  transition  than  in  making  awards  at  lower  cost  to  the  government. 
Selection  preference  is  also  given  to  small  business  proposals.  Selection 
of  large  businesses  is  allowed,  but  only  if  their  offers  for  the  same 
requirement  are  superior  to  those  of  the  small  businesses.  Once  all 
proposals  are  evaluated  and  ranked,  the  source  selection  authority 
makes  the  final  decisions  on  contract  awards.  The  number  of  proposals 
selected  for  contract  award  will  depend  on  the  level  of  RIP  funding 
appropriated  and  the  proposed  project  costs.  The  businesses  are  notified 
of  the  results  and  the  military  departments  and  defense  components  then 
negotiate  and  complete  all  contracting  and  award  procedures  with  the 
businesses.  Contracts  establish  the  cost,  schedule,  and  key  performance 
parameters,  and  deliverables  required  for  projects. 


RIP  Has  Awarded  a  Large 
Number  of  Contracts,  but 
Faced  Administrative 
Challenges  in  Reducing 
Process  Time 


From  fiscal  years  201 1  through  2013,  the  military  departments  and 
defense  components  awarded  $760  million  for  365  projects,  with  89 
percent  of  the  awards  going  to  small  businesses.  Program  officials  said 
that  RIP  attracted  some  new  small  businesses  that  had  not  worked  with 
DOD  previously.  However,  most  of  the  projects  selected  (60-74  percent) 
had  also  received  DOD  funding  through  the  SBIR  program  for  earlier 
technology  development  activities.  Table  1  presents  an  overview  of  the 
numbers  of  proposals,  contract  awards  and  dollar  values,  and  small 
business  participation  rates  for  the  fiscal  year  201 1  through  2014  project 
selection  cycles. 
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Table  1:  Results  of  Rapid  Innovation  Program  Process 


Dollars  in  millions 


Fiscal  year 

2011 

2012 

2013 

2014 

Total 

White  papers 

3,626 

2,405 

2,763 

2,292 

11,086 

Proposals 

268 

126 

235 

149 

778 

Awards 

175 

86 

104 

70-80b 

435-445° 

Value  of  awards 

$370 

$180 

$213 

$165-$170b 

$928-$933c 

Average  award 

$2.2 

$2.1 

$2.1 

$2.1-$2.2b 

N/A 

Awards  to  small  businesses 

93% 

90% 

85% 

88%b 

N/A 

Awards  to  prior  Small 

Business  Innovation  Research 
participants 

74%a 

60% 

63% 

69%b 

N/A 

Legend:  N/A  =  Not  applicable. 

Source:  GAO  analysis  of  DOD  data.  |  GAO-15-421 

Notes: 

aDoes  not  include  Small  Business  Innovation  Research  program  data  from  the  Army,  because  it  is  not 
available. 

bDOD  estimate  or  projection  based  on  historical  fiscal  year  2011-2013  trends  and  information  gained 
from  submission  of  white  paper  proposals  from  offerors  responding  to  the  fiscal  year  2014  Rapid 
Innovation  Program  Broad  Agency  Announcement. 

Includes  fiscal  year  2014  estimates. 

Projects  funded  by  RIP  cover  a  broad  range  of  technologies,  including 
software  tools  and  applications,  measurement  and  testing  devices,  the 
reconfiguration  of  existing  technologies,  and  the  development, 
demonstration,  or  prototyping  of  new  technologies.  Examples  of  projects 
include 

•  software  to  improve  the  rate  and  accuracy  of  the  transmission  of  data, 

•  an  effort  to  improve  the  manufacturing  of  a  component  used  in 
thermal  battery  insulation  technology  to  increase  the  reliability  and  life 
span  of  missile  power  sources,  and 

•  a  compact  water  hand  pump  and  filter  for  purifying  water  on  the 
battlefield. 

According  to  RIP  officials,  although  the  program  is  intended  to  target 
innovative  technologies,  the  department’s  process  enables  the  defense 
components  to  identify  requirements  based  on  evolving  operational  needs 
and  determine  the  kinds  of  projects  to  fund. 
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Since  its  inception,  RIP  has  received  a  high  level  of  interest  from 
businesses— more  than  11,000  white  papers  submitted  in  4  years— which 
has  presented  some  program  administrative  challenges.  In  the  first  year, 
several  of  the  military  departments  and  defense  components  had  to 
“scramble”  to  complete  reviews  and  award  the  contracts  before  program 
funds  expired,  in  part  because  they  did  not  have  sufficient  infrastructure  in 
place  to  administer  the  program.12  According  to  a  RIP  official,  SOCOM 
received  800  of  the  3,600  papers  submitted  that  year  and  was 
overwhelmed  due  to  the  high  volume  of  white  papers.  Since  the  first  year, 
the  components  have  made  adjustments  to  try  and  administer  the 
process  more  efficiently,  such  as  the  move  towards  issuing  a  single 
consolidated  BAA  to  solicit  proposals  in  fiscal  year  2014.  However,  the 
time  needed  to  prepare  the  RIP  implementation  guidelines;  identify 
technology  topics;  prepare  and  execute  the  BAAs;  and  review  and 
evaluate  white  papers  as  well  as  complete  the  other  steps  in  the 
solicitation  and  project  selection  process  has  been  lengthy  in  each 
acquisition  cycle,  taking  about  18  months  to  complete  (see  figure  3). 


12The  period  for  which  the  DOD  Research,  Development,  Test,  and  Evaluation 
appropriations  are  available  for  obligation  is  usually  2  years. 
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Figure  3:  Rapid  Innovation  Program  Acquisition  Cycle 
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Source:  GAO  analysis  of  DOD  data.  |  GAO-15-421 


In  2014,  RIP  program  officials  set  a  qualitative  goal  to  reduce  cycle  time 
for  the  fiscal  year  2014  solicitation  process,  but  said  it  is  challenging  due 
to  the  desire  to  employ  the  two-step  solicitation  process.  For  the  fiscal 
2015  cycle,  they  plan  to  release  the  BAA  earlier  than  in  previous  cycles 
and  have  moved  the  milestone  for  completing  this  acquisition  cycle 
forward  about  30  days. 


Page  14 


GAO-15-421  Rapid  Innovation  Program 


DOD  Uses  a  Variety 
of  Practices  and  Tools 
to  Manage  and 
Oversee  Projects 


The  military  departments  and  defense  components  used  management 
practices  and  tools  to  manage  RIP  projects  similar  to  those  they  use  for 
other  science  and  technology  projects.  Project  managers  and  contracting 
officials  review  progress  reports  submitted  by  contractors  and  maintain 
regular  communications  with  contractors  to  monitor  whether  projects  are 
meeting  cost,  schedule,  and  performance  requirements  specified  in  the 
contracts.  ASD/R&E  established  an  annual  DOD-wide  reporting 
mechanism  in  2013  to  assess  the  technical  performance  of  the  projects, 
and  most  of  them  are  meeting  their  technical  performance  metrics. 

The  FAR  requires  that  contract  quality  assurance  be  performed  at  such 
times  and  places  as  may  be  necessary  to  determine  that  the  goods  or 
services  conform  to  the  contract  requirements.  The  RIP  project  contracts 
describe  the  deliverables,  reporting  schedules,  and  financial  and  technical 
reports  that  the  project  contractor  must  submit.  For  the  40  projects  we 
reviewed,  RIP  project  officials  and  contracting  officer’s  representatives — 
the  technical  subject  matter  expert — conducted  a  variety  of  activities  to 
manage  and  monitor  the  status  of  RIP  projects,  including  continued 
involvement  throughout  contract  implementation  to  support  contractors 
and  help  ensure  that  the  contracted  services  would  be  delivered 
according  to  the  schedule,  cost,  quality,  and  quantity  specified  in  the 
contract. 

RIP  project  officials  said  that  they  manage  and  oversee  projects  in  a 
similar  fashion  as  other  DOD  technology  development  projects.  In 
managing  projects,  officials  said  they  review  progress  and  financial 
reports  submitted  by  the  contractor,  conduct  system  review  meetings 
tailored  to  the  size  and  complexity  of  the  project,  and  engage  in  regular 
communications  with  contractors  through  e-mails,  phone  discussions,  and 
occasional  visits  to  a  contractor’s  facility.  For  example,  on  one  project,  the 
government  team  met  weekly  with  a  contractor  for  the  first  few  months  of 
the  contract  to  review  the  contractor’s  deliverables.  In  addition  to  the 
written  reports,  project  officials  also  conducted  quarterly  program 
management  reviews  which  involved  an  in-person  meeting.  A  project 
official  explained  that  working  with  small  businesses  presented  unique 
challenges  for  oversight,  because  it  requires  the  contractor  to  be 
educated  on  DOD  requirements  such  as  preliminary  design  reviews, 
critical  design  reviews,  and  acceptance  of  testing  plans.  Despite  requiring 
these  extra  education  efforts,  officials  said  that  small  companies  were 
very  flexible  and  responsive  to  oversight  activities.  In  addition  to  the 
project  officials  that  help  manage  the  project,  the  contracting  officer  also 
appointed  a  contracting  officer’s  representative  to  assess  the  contractor’s 
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performance  against  contract  performance  standards  and  to  record  and 
report  this  information. 


Oversight  Of  RIP  Projects  ASD/R&E  established  an  annual  reporting  mechanism  in  2013  to  assess 

the  status  and  progress  of  RIP  projects.  Federal  government  internal 
control  standards  provide  that  effective  and  efficient  control  activities  be 
established  to  enforce  management  directives;  these  include 
performance  measures  and  indicators  to  compare  against  program  goals 
and  objectives.13  We  found  that  RIP  officials  within  DOD  are  taking  these 
steps.  ASD/R&E  collects  information  from  project  officials  on  the 
contractor’s  ability  to  meet  key  technical  performance  parameters,  using 
the  following  three-level  scale  to  assess  the  performance  indicators  for 
each  project: 

•  Green — 80  percent  or  more  of  the  key  performance  parameters, 
goals,  or  thresholds  will  be  met; 

•  Yellow — less  than  80  percent  of  the  key  performance  parameters, 
goals,  or  thresholds  were  being  met;  and 

.  Red  — none  of  the  key  performance  parameters,  goals,  or  thresholds 

were  being  met 

In  the  first  in-process  review,  conducted  in  September  2013,  ASD/R&E 
assessed  the  performance  of  the  175  ongoing  projects  that  were  funded 
in  the  fiscal  year  201 1  acquisition  cycle.  The  review  found  that  86  percent 
of  the  projects  were  likely  to  meet  80  percent  or  more  of  their  key 
performance  parameters  or  goals.  ASD/R&E  conducted  another  in- 
process  review  in  October  2014  and  found  that  85  percent  of  the  fiscal 
year  201 1  projects  and  78  percent  of  the  fiscal  year  2012  projects  were 
likely  to  meet  80  percent  or  more  of  their  key  performance  parameters  or 
goals  (see  figure  4). 


13GAO,  Standards  for  Internal  Control  in  the  Federal  Government ,  GAO/AIMD-OO-21.3.1 
(Washington,  D.C.:  November  1999). 
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Figure  4:  Technical  Performance  for  All  2011  and  2012  Funded  Rapid  Innovation  Program  Projects 


Fiscal  year  2011  projects  (n=175) 


Fiscal  year  2012  projects  (n=86) 


80  percent  or  more  of  the  key  performance  parameters,  goals,  or  thresholds  will  be  met 

Some  will  be  met,  but  less  than  80  percent  of  the  key  performance,  parameters,  goals,  or  thresholds  will  be  met 

None  of  the  key  performance  parameters,  goals,  or  thresholds  will  be  met 


Source:  DOD  data.  |  GAO-15-421 


Note:  Defense  components  include  the  results  for  projects  that  are  not  part  of  the  Air  Force,  Army,  or 
Navy.  These  include,  but  are  not  limited  to:  Defense  Logistics  Agency,  Missile  Defense  Agency,  and 
SOCOM. 


In  addition  to  the  ASD/R&E  in-process  review,  some  military  departments 
and  defense  components  conduct  their  own  performance  reviews.  For 
example,  the  Air  Force  obtains  semi-annual  status  reports  from  project 
managers.  These  status  reports  include  information  associated  with 
technical  risks  that  could  negatively  impact  a  project  or  its  technologies 
and  additional  information  such  as  accomplishments,  planned  actions, 
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and  information  on  projects’  transition  strategies.  The  Navy  requires  its 
system  commands’  chief  technology  officer  or  designee  to  semi-annually 
update  the  project  execution  plan  for  their  projects.14  Project  execution 
plans  include  a  description  of  what  constitutes  a  project’s  transition,  the 
criteria  for  test  events,  and  funding.  This  information  enables  the  Navy  to 
maintain  awareness  of  a  project’s  execution  and  to  provide  assistance 
with  transition  and  eventual  deployment  to  end  users  if  needed.  Similarly, 
the  Missile  Defense  Agency  and  SOCOM  also  perform  periodic  reviews 
of  their  RIP  projects. 


RIP  Has  Transitioned 
Some  Projects,  but  It 
Is  Too  Early  to 
Determine  Overall 
Results 


Some  completed  projects  successfully  transitioned  to  acquisition 
programs  and  other  users,  but  opportunities  may  exist  to  improve  overall 
RIP  transition  outcomes.  DOD  reported  that  half  of  all  fiscal  year  201 1 
projects  (88  of  175)  had  funding  commitments  from  military  users,  which 
indicates  a  likelihood  they  will  transition.  We  also  found  that  half  of  the 
fiscal  year  201 1  completed  projects  we  reviewed  (22  of  44)  had 
technology  that  actually  transitioned  to  an  acquisition  program,  another 
military  user,  a  prime  contractor,  or  was  commercialized.  The  majority  of 
these  projects  had  previously  participated  in  the  SBIR  program.  These 
transition  rates  are  lower  than  what  we  found  in  our  prior  review  of  other 
DOD  technology  transition  programs  that  reported  transition  rates  ranging 
from  about  55  to  85  percent.15  However,  it  is  too  soon  to  accurately 
assess  the  overall  success  of  RIP  due  to  the  limited  number  of  completed 
projects  and  lack  of  established  metrics  to  track  whether  projects  have 
successfully  transitioned  to  users.  We  also  found  several  factors  that  can 
contribute  to  the  transition  success  of  RIP  projects,  such  as  user 
commitment  and  mature  technology  at  the  beginning  of  projects,  which 
were  not  consistently  emphasized  by  all  the  defense  components  in  the 
program. 


14A  project  execution  plan  is  a  tool  which  describes  the  transition  path  fora  project  and 
documents  the  transition  partner’s  agreement.  It  is  similar  to  a  technology  transition 
agreement  used  in  other  transition  programs,  but  it  is  shorter  and  only  needs  approval 
from  a  Navy  RIP  project  manager  and  a  Marine  Colonel,  Navy  Captain,  or  civilian 
equivalent  from  the  program  office. 

15GAO,  Defense  Technology  Development:  Technology  Transition  Programs  Support 
Military  Users,  but  Opportunities  Exist  to  Improve  Measurement  of  Outcomes, 
GAO-13-286  (Washington,  D.C.:  Mar.  7,  2013). 
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DOD  Estimates  Half  of 
2011  Projects  Will  Secure 
Transition  Funding 


As  part  of  its  annual  in  process  review,  DOD  collects  data  from  RIP 
project  managers  on  the  transition  status  of  their  projects.  Based  on  the 
October  2014  review,  DOD  reported  that  half  of  fiscal  year  201 1  projects 
and  41  percent  of  fiscal  year  2012  projects  had  out-year  funding 
committed  by  a  partner  or  user,  which  DOD  uses  as  an  indicator  of  the 
likelihood  transition  will  occur  after  project  completion  (see  figure  5). 


Figure  5:  DOD  Transition  Indicators  for  All  Fiscal  Year  2011  and  2012  Funded  Rapid  Innovation  Program  Projects 


Fiscal  year  2011  projects  (n=1 75) 


Fiscal  year  2012  projects  (n=86) 


Partner  out-year  funding  committed 

Partner  collaboration  identified  but  no  out-year  funding  identified 
No  transition  criteria  or  mechanism  identified 


Source:  DOD  data.  |  GAO-15-421 


Note:  Defense  components  include  the  results  for  projects  that  are  not  part  of  the  Air  Force,  Army,  or 
Navy.  These  include,  but  are  not  limited  to:  Defense  Logistics  Agency,  Missile  Defense  Agency,  and 
SOCOM. 
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Funding  is  a  marker  that  illustrates  the  transition  partner’s  level  of 
commitment  to  transition  the  technology.  A  transition  partner  can  be  a 
program  of  record,  a  prime  contractor,  or  a  user  that  is  willing  to  dedicate 
funding  to  mature  a  technology  past  the  development  stage.  The 
transition  status  of  fiscal  year  201 1  and  2012  projects  varied  across  the 
military  departments  and  defense  components.  At  the  time  of  DOD’s 
review,  many  of  the  fiscal  year  2012  projects  were  ongoing  and  may  be 
up  to  a  year  away  from  completion.  As  a  result,  transition  indicators  for 
fiscal  year  2012  projects  may  not  reflect  final  outcomes.  As  depicted  in 
the  figure,  the  Air  Force  and  the  Navy  reported  that  most  of  their  fiscal 
year  201 1  projects  have  out-year  funding  committed,  but  the  Army  had 
significantly  fewer  projects  with  a  transition  funding  commitment. 
According  to  an  Army  RIP  official,  this  may  be  due  in  part  to  the  Army’s 
selection  of  projects  that  addressed  a  broader  set  of  science  and 
technology  needs  rather  than  needs  defined  by  the  acquisition 
community,  where  partner  funding  commitments  would  be  more  likely. 

For  projects  that  did  not  have  a  commitment  from  a  transition  partner,  the 
military  departments  and  defense  components  reported  that  in  several 
cases,  the  expected  follow-on  acquisition,  procurement  or  support  funds 
were  redirected  to  higher  priorities  or  requirements  for  the  technologies 
were  cancelled  while  the  projects  were  underway. 


Half  of  Completed  Projects 
We  Reviewed 
Successfully  Transitioned 


To  gain  additional  insights  into  the  range  of  transition  outcomes  for  RIP 
projects,  we  assessed  all  52  fiscal  year  201 1  projects  scheduled  for 
completion  through  July  2014,  to  determine  their  transition  results.  We 
found  that  44  of  these  projects  had  been  completed,  meaning  the  contract 
period  of  performance  ended.  Although  projects  are  typically  required  to 
be  completed  within  2  years,  several  of  them  had  encountered  delays  and 
obtained  no-cost  schedule  extensions  of  6  months  or  more  from  RIP 
officials  to  complete  the  remaining  work.  Officials  told  us  extensions  were 
granted  for  various  reasons:  completing  additional  testing;  better  aligning 
projects  with  acquisition  program  schedules;  and  further  demonstrating 
project  technology.  Of  the  44  projects  that  were  completed,  half  had 
technology  that  successfully  transitioned  to  an  acquisition  program,  a 
military  user,  a  prime  contractor,  or  became  a  commercially  available 
product.  Table  2  depicts  these  outcomes,  including  the  range  of  scenarios 
for  projects  that  did  not  transition. 
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Table  2:  Fiscal  Year  2011  Outcomes  for  Rapid  Innovation  Program  Projects 


Completed  (44  projects) 

Not  yet 
completed 

Transitioned 

Not  Transitioned 

Technology 
demonstrated,  but 
no  user 

commitment 

Further 
development  or 
testing  needed 

Demonstration  showed 
technology  not 
competitive  or  failed  to 
meet  requirements 

Army  (n=11) 

3 

2 

2 

3 

1 

Air  Force  (n=9) 

5 

2 

0 

1 

1 

Navy  (n=20) 

10 

4 

2 

1 

3 

Defense  Components 
(n-12) 

4 

4 

0 

1 

3 

DOD  total  (n=52) 

22 

12 

4 

6 

8 

Source:  GAO  analysis  of  DOD  data.  |  GAO-15-421 


Note:  “Completed”  means  that  the  contract’s  period  of  performance  ended.  “Transitioned”  means  that 
the  projects  had  successfully  transitioned  into  an  acquisition  program,  a  military  user,  a  prime 
contractor,  or  became  a  commercially  available  product. 


The  following  are  a  few  specific  examples  of  RIP  projects  that 
successfully  transitioned,  demonstrating  the  array  of  technologies  and  the 
users  supported  by  these  technology  transitions: 

•  Wireless  Vibration  Recorder:  This  Navy  project  provides  a  compact 
and  lightweight  measurement  device  that  can  acquire  data  quickly 
and  easily  on  aircraft,  reducing  flight  test  costs  and  system 
development  time.  The  device  includes  specialized  software  and 
sensors  to  acquire  vibration  and  acceleration  data  for  aircraft  internal 
components  to  determine  why  and  when  components  fail.  The  Navy 
currently  utilizes  a  test  instrumented  aircraft  to  measure  vibrations, 
which  is  time  consuming  and  costly.  RIP  officials  reported  this 
technology  is  expected  to  save  the  Naval  Air  Systems  Command  $3 
million  to  $5  million  in  the  next  4  years  in  flight  test  costs.  This 
technology  is  also  available  commercially. 

•  Enhanced  Ground  Moving  Target  Indicator  (GMTI)-based 
Intelligence,  Surveillance,  and  Reconnaissance:  This  Air  Force 
project  provides  software  applications  to  be  used  by  joint  surveillance 
target  attack  radar  system  operators.  The  applications  are  intended  to 
interact  with  GMTI  data  to  facilitate  pattern  recognition,  automatically 
identify  targets  and  perform  analyses  over  any  set  period  of  time.  For 
example,  instead  of  a  dot  on  the  water,  a  target  could  be  automatically 
identified  as  a  carrier  or  a  person  in  a  rowboat  so  the  analyst  can 
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determine  if  they  need  to  send  a  camera  to  the  area  to  investigate 
further.  RIP  officials  reported  the  Air  Force  Research  Laboratory  has 
installed  the  software  suite  on  a  data  repository  that  can  be  used  by  a 
wide  variety  of  intelligence  analysts. 

•  Multi-Missile  Common  Launch  Tube:  This  United  States  Special 
Operations  Command  project  increases  the  number  of  munitions  that 
can  be  carried  and  launched  from  a  single  common  launch  tube.  This 
technology  doubles  the  number  of  targets  that  can  be  attacked  and 
allows  the  use  of  smaller  warheads  with  precision  delivery,  which  can 
minimize  collateral  damage.  RIP  officials  reported  that  ground  and 
flight  tests  were  successful,  and  the  technology  has  been  transitioned 
into  the  Command’s  common  launch  tube  program. 

•  Automated  Intelligent  Training  with  a  Tactical  Decision-Making 
Serious  Game:  This  Army-funded  project  enhances  a  software  tool 
for  training  Army  officers  in  the  classroom  or  field.  The  enhanced  tool 
improves  upon  a  capability  to  practice  key  leader  cognitive  skills,  such 
as  the  ability  to  rapidly  assess  a  dynamic  situation,  make  sound 
decisions,  and  effectively  direct  subordinate  units  through  scenario- 
based  exercises.  According  to  RIP  officials,  the  software  was 
delivered  to  end  users  at  West  Point  and  future  Army  leaders  are 
using  this  software  to  improve  their  tactical  command  skills. 

•  Mine  Roller  Wheel  Assembly  Improvement:  This  project  improves 
effectiveness  of  the  Marine  Corps  mine  roller  wheel  assembly  over 
the  legacy  system  which  makes  it  more  effective  at  neutralizing 
threats.  A  mine  roller  is  used  to  detonate  and  clear  certain  classes  of 
buried,  pressure-activated  explosive  threats.  This  upgrade  is  expected 
to  be  less  expensive  than  the  cost  of  the  original  roller  wheel 
assembly  it  is  modifying  while  increasing  its  effectiveness  at  a  greater 
range  of  speeds.  In  February  2015  the  Marine  Corps  established  an 
acquisition  program  known  as  the  Wheelbank  Suspension  Upgrade 
Program  to  purchase  the  upgraded  mine  roller  wheel  assembly  and  is 
now  working  toward  a  production  decision  expected  in  the  third 
quarter  of  fiscal  year  2015. 

In  contrast,  we  found  several  different  reasons  projects  did  not  transition. 
DOD  officials  told  us  it  may  take  a  year  or  more  to  transition  a  project 
because  project  completion  may  not  easily  line  up  with  user  timelines  or 
the  DOD  budget  cycle.  A  little  more  than  half  had  successfully 
demonstrated  the  technology  as  planned,  but  had  no  user  commitment. 
According  to  RIP  project  managers,  some  of  the  projects  provided  value 
to  DOD  and  may  yet  transition  in  the  future.  One  of  the  Navy  projects,  for 
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example,  which  provides  a  technology  to  filter  out  interference  in  certain 
military  radios,  has  an  interested  transition  partner,  but  has  not  yet 
transitioned  due  to  funding  constraints.  Officials  reported  they  were 
unable  to  secure  funding  in  the  fiscal  year  2015  and  2016  budgets,  but 
have  submitted  another  request  for  fiscal  year  2017.  DOD  officials  also 
said  that  several  projects  failed  to  transition  due  to  changing  user 
priorities  and  requirements,  such  as  the  phase-out  of  ongoing  missions  in 
Iraq  and  Afghanistan.  Further,  according  to  DOD  officials,  other  projects 
demonstrated  a  proof  of  concept  or  provided  data  to  develop 
requirements,  but  were  not  planned  to  transition  into  a  program  of  record. 
For  example,  one  Army  project  delivered  a  prototype  field  kitchen  that 
demonstrated  potential  capabilities,  and  the  results  will  be  used  to  inform 
future  requirements  for  a  program  of  record  planned  in  2019. 

Other  RIP  projects  that  did  not  transition  needed  further  technology 
development  or  additional  testing — such  as  software  that  needed  security 
accreditation — before  they  could  be  integrated  into  a  DOD  program.  For 
example,  a  Navy-funded  project  for  a  torpedo  array  nose  assembly, 
designed  to  improve  the  array’s  performance  in  shallow  water,  needs 
additional  testing  to  complete  the  array  qualification  effort.  If  successful, 
this  array  nose  assembly  is  expected  to  provide  a  second  source  at  a 
lower  cost  than  the  current  array. 

On  the  other  hand,  officials  told  us  some  of  the  projects  did  not  meet  user 
requirements  or  did  not  demonstrate  that  their  technology  was  better  than 
existing  technology.  For  example,  a  SOCOM-funded  project  to  develop 
an  upgraded  antenna  that  could  operate  in  the  presence  of  jamming  did 
not  meet  a  performance  standard.  Also,  an  Air  Force  project  to  test 
capabilities  of  a  spectral  flare  showed  the  flare  did  not  meet  all  of  the 
requirements  and  did  not  offer  significant  improvement  in  performance 
compared  to  the  current  flare.  Officials  said  that  although  projects  may 
appear  unsuccessful,  in  some  cases  they  provided  valuable  information 
that  could  improve  future  decisions. 
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Mature  Technology,  User 
Commitment,  and 
Program  Support 
Contribute  to  Successful 
Transition;  However,  DOD 
Has  Not  Taken  Steps  to 
Evaluate  This 


Technology  Maturity 


In  our  past  work  on  DOD  technology  transition  activities,  we  found  that 
several  factors  are  important  to  successful  transition,  including  the 
selection  of  projects  that  have  sufficiently  mature  technologies  and  early 
endorsement  from  intended  military  users.  In  addition,  once  projects  are 
selected,  they  require  effective  management  processes  to  support 
technology  transition.16  RIP  officials  and  project  managers  that  we 
interviewed  also  noted  that  these  factors  can  contribute  to  successful 
transition.  In  general,  however,  at  the  DOD  level  there  has  not  been  an 
effort  to  understand  the  extent  to  which  these  factors  may  be  contributing 
to  differences  in  transition  and  to  communicate  lessons  learned  from 
military  departments  or  defense  components  with  a  higher  percentage  of 
transition  success.  Without  such  an  understanding,  DOD  may  be  missing 
opportunities  to  leverage  practices  from  certain  program  components  that 
could  help  improve  overall  transition  success  rates. 

One  high-level  RIP  official  explained  that  a  relatively  mature  technology — 
one  that  has  matured  to  at  least  a  technology  readiness  level  (TRL)  6  at 
the  time  of  the  formal  proposal — involves  less  technical  risk  and  is  in  a 
better  position  to  transition.  TRLs  are  measures  of  the  maturity  level  of  a 
technology  ranging  from  paper  studies  (level  1),  to  prototypes  that  can  be 
tested  in  a  realistic  environment  (level  7),  to  an  actual  system  that  has 
proven  itself  in  mission  operations  (level  9). 17  Program  officials  indicated 
that  sometimes  projects  started  at  lower  TRLs  instead  of  the  desired  level 
6  or  higher.  In  the  projects  we  reviewed,  TRLs  ranged  from  2  to  8  when 
the  project  began,  with  about  half  below  TRL  6.  Although  high-level  DOD 
implementation  guidance  for  RIP  has  not  specified  a  minimum  TRL,  the 
broad  agency  announcements  have  required  projects  to  self-report  their 
TRL  when  applying  to  RIP.  In  an  effort  to  target  more  mature 
technologies,  the  fiscal  year  2014  consolidated  broad  agency 
announcement,  released  in  June  2014,  now  states  that  DOD  seeks  a  TRL 
goal  of  5  to  6  for  entry  and  a  goal  of  7  to  9  for  exit.  Proposals  with  a  lower 
TRL  rating  are  considered  for  award  in  circumstances  of  exceptional 
technical  merit  or  potential  higher  benefit  as  warranted  by  the  Source 
Selection  Authority.  RIP  officials  said  that  since  the  TRLs  are  self- 
reported  by  contractors,  they  may  actually  be  lower  than  stated.  However, 
as  part  of  the  overall  RIP  award  process,  DOD  subject  matter  experts  are 


16GAO-1 3-286. 
17GAO-1 3-286. 
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User  Commitment 


Program  Support 


to  review  the  support  for  the  technical  approach  presented  in  proposal 
submissions,  including  information  on  TRLs. 

We  previously  reported  that  early  project  endorsement  from  intended 
users  and  other  key  stakeholders  in  the  acquisition  community  is 
important  for  project  transition.18  RIP  officials  also  agreed  that  having  a 
transition  partner,  such  as  a  specific  program  of  record,  identified  prior  to 
the  beginning  of  the  project  contributes  to  transition  success.  Other  RIP 
officials  said  user  participation  in  ongoing  project  reviews  further  ensures 
the  final  product  is  aligned  with  user  requirements.  In  the  required 
reporting  to  Congress,  all  of  the  projects  we  reviewed  identified  an 
acquisition  program  or  potential  user  as  the  project  started;  however,  we 
found  the  level  of  user  interest  and  commitment  varied  and  in  some  cases 
changed  during  project  execution. 

The  military  departments  and  defense  components  varied  in  the  approach 
and  level  of  support  they  used  to  manage  RIP  technology  transition. 
According  to  an  OSD-level  RIP  official,  the  Navy  and  the  Air  Force  have 
more  robust  processes  to  support  transition  than  the  other  components. 
For  example,  the  Navy  and  the  Air  Force  provide  additional  internal 
guidance  for  RIP  participants  that  describe  the  roles  and  responsibilities 
of  participants.  These  departments  also  require  project  reviews  every  6 
months  until  project  completion  and  stress  the  importance  of  lessons 
learned  for  RIP  project  managers.  For  example,  the  Air  Force  Life  Cycle 
Management  Center’s  guidance  states  that  analysis  of  RIP  outcomes  at 
the  conclusion  of  each  project  cycle  can  identify  lessons  learned  from 
processes,  successes,  challenges,  and  recommendations  that  can  help 
build  the  program  knowledge  base  and  establish  best  practices  in 
program  implementation  and  stakeholder  relations. 

The  Navy’s  guidance  describes  the  importance  of  quality  control, 
consistent  management,  and  continuous  improvement  to  help  avoid 
demonstrating  a  technology  that  has  nowhere  to  transition.  The  Navy  also 
uses  project  execution  plans  that  describe  transition  plans,  codify 
transition  partner  agreements,  and  describe  criteria  to  evaluate  seminal 
transition  events.19  In  addition,  the  Office  of  Naval  Research’s  risk 


18GAO-1 3-286. 

19The  seminal  transition  event  has  pass/fail  criteria  that  is  used  to  determine  if  the  project 
will  be  integrated  or  purchased. 
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management  team  can  provide  support  for  small  businesses  to  stay  on 
track  in  fulfilling  RIP  contracts,  including  making  sure  companies  can 
ramp  up  production  if  their  projects  are  transitioned.  We  have  previously 
reported  that  this  office  has  a  well-established  technology  transition  focus 
which  may  contribute  to  project  success.  Because  of  this,  the  Navy  may 
be  better  aware  of  the  benefits  and  obstacles  associated  with  a 
substantial  portion  of  their  S&T  portfolio.20  This  knowledge  can  better 
inform  investment  decisions  made  by  Navy  leadership. 


RIP  Does  Not  Track  Final  While  the  program  has  established  metrics  to  determine  whether  projects 
Transition  Outcomes  have  a  funding  commitment  from  users  and  are  therefore  likely  to 

transition,  it  does  not  track  the  degree  to  which  completed  projects  have 
actually  transitioned.  RIP  officials  view  transition  success  broadly,  as  a 
technology  which  is  inserted  into  an  acquisition  program  of  record, 
incorporated  into  a  weapon  system  manufacturing  process,  adopted  for 
use  by  a  depot  or  logistics  center,  or  available  for  purchase  on  the 
General  Services  Administration  federal  supply  schedule  or  in  the 
commercial  market.  However,  RIP  does  not  formally  track  projects 
beyond  completion  and  whether  they  are  inserted  into  acquisition 
programs  or  used  for  other  purposes,  which  limits  DOD’s  ability  to  know 
the  final  transition  outcomes  and  whether  any  benefits  were  achieved.  As 
we  have  found  in  the  past,  tracking  and  measuring  technology  transition 
and  the  impact  of  those  transitions,  such  as  cost  savings  or  deployment 
of  a  technology  into  a  weapon  system,  provides  key  feedback  that  can 
inform  the  management  of  programs  such  as  RIP.  We  previously 
recommended  the  Secretary  of  Defense  require  all  technology  transition 
programs  in  the  department  to  track  and  measure  project  outcomes — 
including  long  term  benefits  for  acquisition  programs  or  users.21  DOD 
generally  agreed  with  our  recommendations  but  we  found  no  evidence 
that  DOD  has  taken  any  action  yet;  this  recommendation  would  also 
apply  to  RIP,  as  it  is  a  technology  transition  program.  In  addition,  DOD 
has  not  established  an  overall  transition  goal  for  RIP,  so  it  is  not  clear 
what  is  expected  in  terms  of  success.  Federal  government  internal  control 
standards  require  that  effective  and  efficient  control  activities  be 
established  which  include  performance  measures  and  indicators  to 


20GAO-1 3-286. 
21GAO-1 3-286. 
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compare  against  program  goals  and  objectives.22  Furthermore,  although 
DOD  annually  provides  a  report  to  Congress  on  the  number  and 
description  of  projects  that  are  funded  through  RIP,  the  department  is  not 
required  to  provide  information  on  transition  results.  Without  this 
information,  Congress  lacks  insight  about  the  program’s  performance 
which  is  important  for  conducting  program  oversight. 

In  our  discussions  with  DOD  RIP  officials,  they  estimated  that  about  50 
percent  or  more  of  the  RIP  projects  will  transition  when  completed, 
although  they  said  it  is  too  early  to  accurately  assess  the  overall 
effectiveness  of  the  program.  Some  officials  indicated  that  there  are  not 
enough  projects  completed  and  it  could  take  two  or  more  years  after  a 
project  is  completed  before  it  is  successfully  transitioned  and  used.  In 
addition,  projects  awarded  in  RIP’s  first  acquisition  cycle  may  not 
accurately  reflect  program  performance  for  the  following  years,  in  part 
because  there  were  implementation  challenges  associated  with  starting  a 
new  program.  Therefore,  more  data  that  captures  final  project  outcomes 
and  long-term  program  experience  is  required  to  accurately  assess 
performance.  In  our  prior  review  of  DOD  technology  transition  programs 
that  provide  structured  mechanisms  and  funding  to  facilitate  technology 
transition,  we  found  that  programs  reported  rates  of  technology  transition 
ranging  from  about  55  to  85  percent.23  RIP  officials’  estimate  of  50 
percent  is  at  the  lower  end  of  this  range,  as  is  experience  to  date  with  the 
transition  of  completed  projects. 


Conclusions 


RIP  is  a  relatively  new  program  established  by  Congress  to  identify  and 
transition  innovative  technologies  to  support  the  warfighter.  The  authority 
to  carry  out  the  program  will  terminate  at  the  end  of  this  fiscal  year,  so  it  is 
important  to  determine  whether  it  is  accomplishing  its  intended  objective 
and  worth  continuing.  The  RIP  projects  we  reviewed  have  had  some 
success  in  transitioning  technologies  to  acquisition  programs  and  other 
users,  but  too  few  have  been  completed  to  determine  the  extent  of 
technology  transition  occurring  and  level  of  benefit  they  provide  to  the 
warfighter.  However,  DOD  has  not  established  specific  transition 
performance  targets  for  RIP,  does  not  track  project  outcomes  beyond 
whether  partner  out-year  funding  has  been  committed,  and  does  not 


22GAO/AIMD-00-21 .3.1 . 
23  GAO-13-286. 
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report  program  results  to  Congress,  as  this  is  not  currently  required. 
Consequently,  without  an  overall  transition  performance  target  and  better 
measure  of  outcomes,  it  is  unclear  whether  RIP  has  been  successful  at 
transitioning  innovative  technologies  and  is  worth  continuing.  We  continue 
to  believe  that  the  recommendation  we  made  in  2013 — for  DOD  to  track 
and  measure  the  outcomes  of  its  numerous  technology  transition 
programs  in  order  to  improve  the  visibility  and  management  of  these 
efforts — has  merit  and  is  applicable  to  RIP. 

In  addition,  although  the  military  departments  and  defense  components 
have  implemented  a  structured  process  for  soliciting  projects,  they  do  not 
always  select  projects  that  have  a  high  likelihood  of  successfully 
transitioning,  such  as  those  with  mature  technologies  or  commitments 
from  transition  partners.  DOD  components  are  capitalizing  on  these 
success  factors  to  varying  degrees.  The  Air  Force  and  Navy,  in  particular, 
incorporate  them  in  RIP  projects  and,  as  a  result,  appear  to  be  realizing 
significantly  higher  transition  rates  for  their  projects,  even  in  the  program’s 
early  years.  Without  a  more  consistent  focus  on  these  factors  during 
project  selection,  opportunities  to  achieve  higher  levels  of  transition 
success  for  RIP  may  remain  limited. 


Matter  for 

Congressional 

Consideration 


Congress  should  consider,  if  it  decides  to  re-authorize  RIP,  requiring 
DOD  to  submit  annual  reports  to  Congress  on  the  transition  results  of  the 
program  to  improve  accountability  and  transparency  of  the  program. 


Recommendations  for  ^  Congress  re-authorizes  RIP  then,  to  improve  visibility  and  management 

of  DOD’s  ability  to  transition  technologies  through  the  program,  we 
Executive  Action  recommend  that  the  Secretary  of  Defense  direct  the  Under  Secretary  of 

Defense  for  Acquisition,  Technology,  and  Logistics  to  take  the  following 
two  actions: 

•  Establish  an  overall  technology  transition  goal  for  RIP;  and 

•  Identify  and  apply  factors  that  contribute  to  the  likelihood  of 
technology  transition  success  more  consistently  across  the  program. 


Page  28 


GAO-15-421  Rapid  Innovation  Program 


Agency  Comments 
and  our  Evaluation 


We  provided  a  draft  of  this  report  to  DOD  for  comment.  In  its  written 
comments,  DOD  disagreed  with  our  first  recommendation  and  concurred 
with  the  second  recommendation.  DOD’s  comments  are  reproduced  in 
appendix  II. 

DOD  disagreed  with  our  first  recommendation,  to  establish  an  overall 
technology  transition  goal  for  RIP.  The  department  stated  that 
establishing  a  transition  goal  will  impede  the  program’s  objective  of 
encouraging  innovative  technologies  in  defense  programs.  In  addition,  the 
department  raised  the  concern  that,  since  only  a  limited  number  of 
projects  are  complete,  it  is  too  soon  to  accurately  assess  the  overall 
success  of  RIP  and  therefore,  establishing  a  goal  is  not  in  the  best 
interest  of  the  program.  Further,  DOD  said  that  in  line  with  the  Secretary’s 
release  of  the  Defense  Innovation  Initiative  in  November  2014 — a 
department  initiative  to  pursue  innovative  ways  to  sustain  and  advance 
military  superiority — it  needs  to  maintain  flexibility  in  RIP  to  address  risky 
technical  requirements  that  may  not  be  mature  enough  to  transition  to 
acquisition  programs,  but  may  present  opportunities  for  prototyping, 
experimentation,  or  innovative  test  and  evaluation.  However,  the 
department  said  it  would  continue  to  measure  and  assess  program 
transition  results  annually. 

We  continue  to  believe  that  it  is  important  for  DOD  to  establish  specific 
transition  performance  targets  for  RIP  and  assess  the  extent  to  which  the 
program  is  successfully  transitioning  technologies  to  support  acquisition 
programs.  While  we  agree  that  flexibility  may  be  necessary  at  times  in 
RIP  to  address  risky  technologies  that  may  not  be  mature  enough  for 
acquisition  programs,  the  purpose  of  the  program  is  to  target  innovative 
technologies  that  have  the  potential  to  support  acquisition  programs  in  the 
near  term.  We  believe  that  there  are  many  other  technology  development 
programs  and  activities  within  the  department’s  science  and  technology 
enterprise  with  broader  objectives  than  RIP  and  more  closely  align  with 
the  goal  of  the  Defense  Innovation  Initiative.  Furthermore,  as  we  pointed 
out  in  this  report,  DOD  has  historically  experienced  problems  in 
transitioning  technologies  out  of  its  science  and  technology  enterprise 
and  into  acquisition  programs,  and  RIP  was  established  as  one 
mechanism  aimed  at  facilitating  transition.  Establishing  a  goal  will  not 
impede,  but  instead  help  focus  efforts  on  meeting  this  objective.  We 
recognize  it  is  early  in  the  program  and  that  transition  goals  may  need  to 
be  adjusted  as  the  program  matures. 

The  department  agreed  on  the  need  to  identify  and  apply  factors  that 
contribute  to  the  likelihood  of  technology  transition  success  more 
consistently  across  the  program.  Their  response  identified  several  actions 
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already  taken,  and  indicated  that  if  Congress  reauthorizes  RIP  the 
department  will  continue  to  identify  additional  best  practices  that 
contribute  to  transition  success. 


We  are  sending  copies  of  this  report  to  the  appropriate  congressional 
committees,  the  Secretary  of  Defense,  the  Under  Secretary  of  Defense 
for  Acquisition,  Technology,  and  Logistics,  and  other  interested  parties.  In 
addition,  the  report  is  available  at  no  charge  on  the  GAO  website  at 

http://www.gao.gov. 

If  you  or  your  staff  have  any  questions  about  this  report,  please  contact 
me  at  (202)  512-4841  or  mackinm@gao.gov.  Contact  points  for  our 
Offices  of  Congressional  Relations  and  Public  Affairs  may  be  found  on 
the  last  page  of  this  report.  GAO  staff  who  made  key  contributions  to  this 
report  are  listed  in  appendix  III. 


Michele  Mackin 
Director 

Acquisition  and  Sourcing  Management 
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Appendix  I:  Objectives,  Scope,  and 
Methodology 


The  objectives  of  this  review  were  to  assess  the  extent  to  which  (1)  the 
Department  of  Defense  (DOD)  has  established  a  competitive  and  merit 
based  process  to  solicit  and  award  Rapid  Innovation  Program  (RIP) 
contracts,  (2)  DOD  has  established  practices  to  manage  and  oversee  the 
execution  of  RIP  projects,  and  (3)  RIP  is  meeting  its  objective  of  rapidly 
inserting  innovative  technologies  in  defense  acquisition  programs. 

To  assess  how  DOD  solicited  and  awarded  RIP  projects,  we  examined 
DOD  and  military  department  policies  and  guidance  on  the  program  such 
as  DOD’s  annual  implementation  guidelines,  any  internal  guidance 
developed  by  the  military  departments  and  defense  agencies  participating 
in  the  program,  and  other  policy  documents.  For  purposes  of  this  report, 
we  defined  competitive  awards  as  those  using  the  competitive  procedures 
listed  in  Federal  Acquisition  Regulation  (FAR)  Subpart  6.1  (Full  and  Open 
Competition).  Also,  we  used  FAR  Part  35  (Research  and  Development 
Contracting)  to  identify  the  policies  and  procedures  for  using  broad 
agency  announcements.  We  analyzed  the  broad  agency  announcements 
issued  between  201 1  and  2014  by  the  military  departments  and  for  the 
defense  components.  For  a  sample  of  40  projects  awarded  using  fiscal 
year  201 1  and  2012  funding,  we  examined  source  selection  documents 
such  as  project  white  papers,  proposals,  and  project  scoring  sheets  used 
in  the  source  selection  process.  We  interviewed  officials  responsible  for 
developing  the  guidance,  RIP  leads  for  each  of  the  services  and 
components,  and  those  involved  in  the  source  selection  process.  In 
addition,  we  reviewed  the  Federal  Acquisition  Regulation  and  DOD 
Source  Selection  Procedures.  The  results  from  the  selected  40  projects 
cannot  be  generalized  to  all  RIP  projects,  but  provide  valuable  insight. 

To  assess  whether  DOD  has  established  practices  to  manage  and 
oversee  the  execution  of  RIP  projects,  we  reviewed  a  nongeneralizable 
random  sample  of  40  RIP  projects  across  DOD  that  were  awarded  with 
fiscal  year  201 1  and  2012  funding.  The  sample  included  20  projects  from 
each  fiscal  year,  with  5  projects  from  each  of  the  military  departments  and 
the  defense  agency  component.  For  these  projects,  we  reviewed  project 
contracts,  contractor  reports,  and  agency  project  assessments  and 
technical  performance  data.  We  also  interviewed  officials  responsible  for 
high-level  project  oversight  and  management  such  as  program  leads  for 
the  services  and  defense  agency  components  in  addition  to  contracting 
officers  and  contracting  officer’s  representatives  responsible  for  day-to- 
day  management  activities.  Further,  we  reviewed  data  from  DOD’s  in- 
process  review  on  the  technical  performance  of  ongoing  RIP  projects.  We 
did  not  independently  assess  the  accuracy  of  technical  performance  data, 
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Appendix  I:  Objectives,  Scope,  and 
Methodology 


but  reviewed  and  discussed  the  data  with  DOD  subject  matter  experts 
and  determined  it  was  sufficiently  reliable  for  the  purposes  of  this  report. 

To  determine  whether  the  RIP  is  meeting  its  goal  of  rapidly  inserting 
innovative  technologies  into  acquisition  programs,  we  reviewed  available 
program  monitoring  information  and  assessed  the  transition  status  of  all 
projects  that  were  scheduled  for  completion  by  the  end  of  July  2014, 
which  included  52  projects  that  were  awarded  with  fiscal  year  201 1 
funding.1  For  these  projects,  we  interviewed  RIP  project  officials  and 
collected  and  analyzed  documents  outlining  processes  and  procedures 
used  to  manage  the  projects  and  promote  transition  opportunities.  In 
these  interviews,  we  discussed  whether  projects  successfully  transitioned 
and  factors  that  may  have  helped  or  hindered  project  execution  and/or 
successful  transition  to  defense  acquisition  programs  or  other  military 
users.  We  also  reviewed  prior  GAO  studies  on  DOD  technology  transition 
and  best  practices  for  transition  to  identify  what  practices  may  facilitate 
technology  transition.  Further,  we  reviewed  data  from  DOD  in-process 
reviews  on  the  status  of  RIP  projects  which  provided  information  on  the 
likelihood  of  fiscal  year  201 1  and  2012  projects  transitioning.  We 
assessed  the  accuracy  of  the  transition  performance  data  from  DOD’s  in- 
process  review  by  comparing  its  results  to  the  52  projects  that  we 
examined.  We  determined  the  data  from  the  in-process  review  was 
sufficiently  reliable  for  the  purposes  of  this  report. 

We  conducted  this  performance  audit  from  April  2014  to  May  2015  in 
accordance  with  generally  accepted  government  auditing  standards. 
Those  standards  require  that  we  plan  and  perform  the  audit  to  obtain 
sufficient,  appropriate  evidence  to  provide  a  reasonable  basis  for  our 
findings  and  conclusions  based  on  our  audit  objectives.  We  believe  that 
the  evidence  obtained  provides  a  reasonable  basis  for  our  findings  and 
conclusions  based  on  our  audit  objectives. 


Although  Congress  first  appropriated  funds  for  RIP  in  fiscal  year  201 1 ,  projects  were  not 
awarded  until  the  end  of  fiscal  year  2012  and  then  not  expected  to  be  completed  until  up 
to  two  years  after  contract  award,  which  was  the  end  of  fiscal  year  2014. 
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of  Defense 


RESEARCH 
AND  ENGINEERING 


ASSISTANT  SECRETARY  OF  DEFENSE 

3030  DEFENSE  PENTAGON 
WASHINGTON.  DC  20301-3030 


APR  17  2015 


Ms.  Michele  Mackin 

Director,  Acquisition  and  Sourcing  Management 
U.S.  Government  Accountability  Office 
441  G  Street,  N.W. 

Washington.  DC  20548 

Dear  Ms.  Mackin: 

This  is  the  Department  of  Defense  (DoD)  response  to  the  GAO  Draft  Report,  GAO-15- 
421,  “DOD  RAPID  INNOVATION  PROGRAM:  Some  Technologies  Have  Transitioned  to 
Military  Users,  but  Steps  Can  Be  Taken  to  Improve  Program  Metrics  and  Outcomes,”  dated 
March  17,  2015  (GAO  Code  100029).  Detailed  comments  on  the  report  recommendations  are 
enclosed. 


Sincerely, 


Enclosure: 
As  stated 


Page  33 


GAO-15-421  Rapid  Innovation  Program 


Appendix  II:  Comments  from  the  Department 
of  Defense 


GAO  Draft  Report  Dated  March  17,  2015 
GAO-15-421  (GAO  CODE  100029) 

“DOD  RAPID  INNOVATION  PROGRAM:  SOME  TECHNOLOGIES  HAVE 
TRANSITIONED  TO  MILITARY  USERS,  BUT  STEPS  CAN  BE  TAKEN  TO  IMPROVE 
PROGRAM  METRICS  AND  OUTCOMES” 

DEPARTMENT  OF  DEFENSE  COMMENTS 
TO  THE  GAO  RECOMMENDATION 


RECOMMENDATION  1:  If  Congress  re-authorizes  Rapid  Innovation  Program  (RIP)  then,  to 
improve  visibility  and  management  of  DOD's  ability  to  transition  technologies  through  the 
program,  GAO  recommends  that  the  Secretary  of  Defense  direct  the  Under  Secretary  of  Defense 
for  Acquisition,  Technology  and  Logistics  to  establish  an  overall  technology  transition  goal  for 
RIP. 

DOD  RESPONSE:  Non-concur. 

Assuming  Congress  re-authorizes  RIP,  establishing  an  overall  transition  goal  will  impede  the 
congressional,  and  Department’s  objective  to  encourage  new  innovation  in  defense  programs, 
particularly  from  small  businesses.  As  provided  in  statute,  the  purpose  of  RIP  is  to  stimulate 
innovative  technologies  pursuant  to  phase  II  Small  Business  Innovative  Research  Program 
projects,  technologies  developed  by  defense  laboratories,  and  other  innovative  technologies. 
Also,  as  noted  by  the  GAO,  only  a  limited  number  of  projects  are  complete,  so  if  s  too  soon  to 
accurately  assess  the  overall  success  of  RIP,  and  therefore,  establishing  a  transition  goal  is  not  in 
the  best  interest  of  the  program. 

With  the  Secretary’s  release  of  the  Defense  Innovation  Initiative  in  November  2014,  we  are 
challenged  to  identify  revolutionary  and  potentially  disruptive  technologies  and  link  these 
capabilities  to  counter  operational  threats.  Accordingly,  we  need  to  retain  the  flexibility  in  RIP 
to  address  risky  technical  requirements  that  may  not  be  mature  enough  to  satisfy  cost,  schedule 
or  performance  requirements  for  acquisition  programs;  but  may  be  opportunities  for  prototyping, 
experimentation,  or  innovative  test  and  evaluation. 

The  Department  will  continue,  however,  to  measure  and  assess  transition  results  annually,  and 
can  make  results  available,  as  needed.  For  example,  based  on  the  Department's  review  of  the 
261  projects  awarded  from  the  fiscal  year  (FY)  2011  and  2012  RIP  funds,  about  47  percent,  or 
123  projects  have  transitioned  or  are  currently  scheduled  to  transition  to  a  program  of  record  or 
other  defense  acquisition  customer  when  complete.  This  is  consistent  with  the  GAO's  findings 
that  50  percent  of  FY  2011  projects  have  out-year  funding  commitments  from  military  users  to 
facilitate  transition.  We  expect  the  transition  results  will  improve  as  the  FY  2013  and  2014 
projects  are  executed  and  completed,  as  the  defense  components  improve  transition  practices 
with  small  business  performers  and  more  closely  link  RIP  requirements  to  acquisition  Program 
Executive  Offices  (PEOs),  Program  Managers  (PMs),  and  other  technology  customers  or  users. 
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Appendix  II:  Comments  from  the  Department 
of  Defense 


RECOMMENDATION  2:  If  Congress  re-authorizes  RIP  then,  to  improve  visibility  and 
management  of  DOD’s  ability  to  transition  technologies  through  the  program,  GAO  recommends 
that  the  Secretary  of  Defense  direct  the  Under  Secretary  of  Defense  for  Acquisition,  Technology 
and  Logistics  to  identify  and  apply  factors  that  contribute  to  the  likelihood  of  technology 
transition  success  more  consistently  across  the  program. 

DOD  RESPONSE:  Concur. 

The  Department  has  already  identified  a  number  of  actions  and  issued  guidelines  that  contribute 
to  the  likelihood  of  technology  transition,  e.g. : 

•  Ensuring  the  PEO/PM  acquisition  representatives  or  other  users  of  the  RIP  technology 
participate  on  the  source  selection  evaluation  teams  to  assess  offeror  proposals. 

•  Assessing  transition  probability  as  part  of  evaluating  an  offeror’s  technical  approach. 

The  evaluation  criteria  provided  in  the  RIP  FY  2014  and  2015  Broad  Agency 
Announcements  include  an  assessment  of  the  probability  for  transition  of  a  RIP  proposal 
into  an  acquisition  program,  a  military  system,  or  other  military  capability. 

•  Establishing  guidelines  for  using  existing  procurement  authorities  to  expedite  further 
development,  production,  and  fielding  of  successful  RIP  projects.  This  includes 
providing  options  in  RIP  contracts  for  further  research,  testing,  low  rate  initial 
production,  or  production  for  funding  by  acquisition  programs  of  record  or  other 
technology  customers. 

If  Congress  re-authorizes  RIP,  we  will  identify  additional  best  practices  for  transition  and  request 
the  USD/AT&L  approve  guidelines  that  further  contribute  to  transition  success  across  the 
program. 
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GAO’s  Mission 

The  Government  Accountability  Office,  the  audit,  evaluation,  and 
investigative  arm  of  Congress,  exists  to  support  Congress  in  meeting  its 
constitutional  responsibilities  and  to  help  improve  the  performance  and 
accountability  of  the  federal  government  for  the  American  people.  GAO 
examines  the  use  of  public  funds;  evaluates  federal  programs  and 
policies;  and  provides  analyses,  recommendations,  and  other  assistance 
to  help  Congress  make  informed  oversight,  policy,  and  funding  decisions. 
GAO’s  commitment  to  good  government  is  reflected  in  its  core  values  of 
accountability,  integrity,  and  reliability. 

Obtaining  Copies  of 
GAO  Reports  and 
Testimony 

The  fastest  and  easiest  way  to  obtain  copies  of  GAO  documents  at  no 
cost  is  through  GAO’s  website  (http://www.gao.gov).  Each  weekday 
afternoon,  GAO  posts  on  its  website  newly  released  reports,  testimony, 
and  correspondence.  To  have  GAO  e-mail  you  a  list  of  newly  posted 
products,  go  to  http://www.gao.gov  and  select  “E-mail  Updates.” 

Order  by  Phone 

The  price  of  each  GAO  publication  reflects  GAO’s  actual  cost  of 
production  and  distribution  and  depends  on  the  number  of  pages  in  the 
publication  and  whether  the  publication  is  printed  in  color  or  black  and 
white.  Pricing  and  ordering  information  is  posted  on  GAO’s  website, 
http://www.gao.gov/ordering.htm. 

Place  orders  by  calling  (202)  512-6000,  toll  free  (866)  801-7077,  or 

TDD  (202)  512-2537. 

Orders  may  be  paid  for  using  American  Express,  Discover  Card, 
MasterCard,  Visa,  check,  or  money  order.  Call  for  additional  information. 

Connect  with  GAO 

Connect  with  GAO  on  Facebook,  Flickr,  Twitter,  and  YouTube. 

Subscribe  to  our  RSS  Feeds  or  E-mail  Updates.  Listen  to  our  Podcasts. 
Visit  GAO  on  the  web  at  www.gao.gov. 

To  Report  Fraud, 
Waste,  and  Abuse  in 
Federal  Programs 

Contact: 

Website:  http://www.gao.gov/fraudnet/fraudnet.htm 

E-mail:  fraudnet@gao.gov 

Automated  answering  system:  (800)  424-5454  or  (202)  512-7470 

Congressional 

Relations 

Katherine  Siggerud,  Managing  Director,  siggerudk@gao.gov,  (202)  512- 
4400,  U.S.  Government  Accountability  Office,  441  G  Street  NW,  Room 
7125,  Washington,  DC  20548 

Public  Affairs 

Chuck  Young,  Managing  Director,  youngd@gao.gov,  (202)  512-4800 

U.S.  Government  Accountability  Office,  441  G  Street  NW,  Room  7149 
Washington,  DC  20548 
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